Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[Gluon] Improve estimator usability and fix logging logic #16810

Merged
merged 6 commits into from
Nov 16, 2019

Conversation

leezu
Copy link
Contributor

@leezu leezu commented Nov 14, 2019

Description

gluon.contrib.estimator used a global Logger obtained via
logging.getLogger('gluon.contrib.estimator.event_handlers'). This logger used
to be configured every time a gluon.contrib.estimator.LoggingHandler was
created, which is a bug. We can't modify a global Logger instance whenever the
user creates an Estimator and a LoggingHandler.

Instead, this commit separates the LoggingHandler (responsible for logging
metadata during estimator.fit) from the configuration of the Logger.

We expose the Logger as attribute of the Estimator, and configure it to output
to stdout by default. Instructions are given how users can configure the
Estimator.logger to log to a file instead.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Comments

  • This change is backward incompatible, but is important to make. It only changes experimental APIs. The reason is that the previous API design is inconsistent and not user friendly. It is inconsistent, because a global object is modified whenever a LoggingHandler object is created; this leads to buggy behaviour if the user creates multiple LoggingHandlers, as all of them will share the same underlying Python Logger. Instead, we should use a separate logger for each estimator.
  • This change improves usability, by switching the default log stream from sys.stderr to sys.stdout. The following screenshot highlights the need for this.

image

Compare this to

image

gluon.contrib.estimator used a global Logger obtained via
`logging.getLogger('gluon.contrib.estimator.event_handlers')`. This logger used
to be configured every time a gluon.contrib.estimator.LoggingHandler was
created, which is a bug. We can't modify a global Logger instance whenever the
user creates an Estimator and a LoggingHandler.

Instead, this commit separates the LoggingHandler (responsible for logging
metadata during estimator.fit) from the configuration of the Logger.

We expose the Logger as attribute of the Estimator, and configure it to output
to stdout by default. Instructions are given how users can configure the
Estimator.logger to log to a file instead.
Copy link
Member

@roywei roywei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the improvement!

pylint:

************* Module mxnet.gluon.contrib.estimator.event_handler

python/mxnet/gluon/contrib/estimator/event_handler.py:22:0: W0611: Unused import logging (unused-import)

python/mxnet/gluon/contrib/estimator/event_handler.py:24:0: W0611: Unused import sys (unused-import)

@leezu leezu merged commit 93f338d into apache:master Nov 16, 2019
@leezu leezu deleted the estimator branch November 16, 2019 13:20
@leezu
Copy link
Contributor Author

leezu commented Nov 16, 2019

@ptrendx can we add this change to the 1.6 branch? Should I open a PR or will you merge it similar to #16832 ?

@ptrendx
Copy link
Member

ptrendx commented Nov 17, 2019

@leezu yes, go ahead and make a PR to 1.6.x branch

leezu added a commit to leezu/mxnet that referenced this pull request Nov 18, 2019
gluon.contrib.estimator used a global Logger obtained via
`logging.getLogger('gluon.contrib.estimator.event_handlers')`. This logger used
to be configured every time a gluon.contrib.estimator.LoggingHandler was
created, which is a bug. We can't modify a global Logger instance whenever the
user creates an Estimator and a LoggingHandler.

Instead, this commit separates the LoggingHandler (responsible for logging
metadata during estimator.fit) from the configuration of the Logger.

We expose the Logger as attribute of the Estimator, and configure it to output
to stdout by default. Instructions are given how users can configure the
Estimator.logger to log to a file instead.
@leezu leezu mentioned this pull request Nov 18, 2019
ptrendx pushed a commit that referenced this pull request Nov 19, 2019
…16846)

gluon.contrib.estimator used a global Logger obtained via
`logging.getLogger('gluon.contrib.estimator.event_handlers')`. This logger used
to be configured every time a gluon.contrib.estimator.LoggingHandler was
created, which is a bug. We can't modify a global Logger instance whenever the
user creates an Estimator and a LoggingHandler.

Instead, this commit separates the LoggingHandler (responsible for logging
metadata during estimator.fit) from the configuration of the Logger.

We expose the Logger as attribute of the Estimator, and configure it to output
to stdout by default. Instructions are given how users can configure the
Estimator.logger to log to a file instead.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants